An adaptive gradient sampling algorithm for non-smooth optimization

We present an algorithm for the minimization of f : Rn → R, assumed to be locally Lipschitz and continuously differentiable in an open dense subset D of Rn. The objective f may be nonsmooth and/or nonconvex. The method is based on the gradient sampling algorithm (GS) of Burke, Lewis, and Overton [SIAM J. Optim., 15 (2005), pp. 751-779]. It differs, however, from previously proposed versions of GS in that it is variable-metric and only O(1) gradient evaluations are required per iteration. Numerical experiments illustrate that the algorithm is much more efficient than GS in that it consistently requires significantly fewer gradient evaluations. In addition, the adaptive sampling procedure allows for warm-starting of the quadratic subproblem solver so that the number of subproblem iterations per nonlinear iteration is also reduced. Global convergence of the algorithm is proved assuming that the Hessian approximations are positive definite and bounded, an assumption shown to be true for the proposed Hessian approximation updating strategies.